AMENDMENTS TO THE CLAIMS 



This listing of claims will replace all prior versions and listings of claims in the 
application. 

Listing of Claims 

Claim 1 . (Original) A system for evaluating and selecting competition winners from a pool 
of candidates in a competition comprising: 

a database for storing candidate documents; 

a grouping subsystem coupled to said database for grouping said candidate documents 
into candidate data sets; 

a qualification subsystem coupled to said database for determining said candidate data 
sets to be qualified or disqualified; 

a pool reduction subsystem coupled to said database for determining whether said 
candidate data sets having been qualified meet a discretionary eligibility standard, said 
discretionary eligibility standard varying as a function of the number of said candidate data 
sets having been qualified; 

an evaluation subsystem coupled to said qualification subsystem and said database for 
evaluating each of said candidate data sets having met said discretionary eligibility standard 
and scoring one or more evaluation variables in response to said evaluation; and 

a selection subsystem coupled to said evaluation subsystem and database for selecting 
one or more competition finalists responsive to said evaluation and scoring of said candidate 
data sets having met said discretionary eligibility standard. 

Claim 2. (Original) A system as in claim 1 further comprising: a document reception 
subsystem comprising: a network interface module coupled to said database for receiving 
electronic submissions, said database storing said electronic submissions as candidate 
documents, a scanner interface module for interfacing said system to a scanner and receiving 
scans of submitted hardcopy documents, and a data extraction module coupled to said 
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scanner interface module and said database for extracting data from said scans, said database 
storing said extracted data as candidate documents. 

Claim 3. (Original) A system as in claim 2, further comprising: a duplicate checking 
module coupled to said network interface module, said data extraction module and said 
database for determining whether said candidate documents exist in said database, said 
candidate documents having been determined not to exist in said database being stored in said 
database. 

Claim 4. (Original) A system as in claim 1 wherein said qualification subsystem 
comprises: a first filter module coupled to said database for determining whether said 
candidate data sets meet a data completeness standard. 

Claim 5. (Original) A system as in claim 4 wherein said data completeness standard 
requires said candidate data sets to include a minimum set of document types. 

Claim 6. (Original) A system as in claim 4 wherein said evaluation subsystem comprises: 
an evaluation workstation coupled to said database for facilitating evaluation of said 
candidate data sets having met said discretionary eligibility standard, said evaluation 
workstation comprising: a display for displaying information extracted from said candidate 
data sets, and an input device for entering information; a matching module coupled to said 
database for selecting a possible evaluator to evaluate one of said candidate data sets having 
met said discretionary eligibility standard; an evaluator eligibility module coupled to said 
database for determining whether said evaluator is permitted to evaluate said one of said 
candidate data sets having met said discretionary eligibility standard; and an evaluation 
module coupled to said evaluator eligibility module, said database, and said evaluation 
workstation for providing on said display information extracted from said one of said 
candidate data sets having met said discretionary eligibility standard, providing one or more 
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evaluation variables for scoring, and receiving a score for each of said one or more evaluation 
variables from said input device. 

Claim 7. (Original) A system as in claim 1 wherein said selection system comprises: a 
phase 1 selection module coupled to said evaluation subsystem for selecting a first set of 
candidate finalists responsive to total scores generated responsive to corresponding said 
received scores, said first set of candidate finalists not exceeding a predetermined total 
number; said phase 1 selection module further outputting when said first set of candidate data 
sets is less than said predetermined number a sub-pool of said candidate data sets all having 
received equal corresponding said total scores; and a phase 2 selection module coupled to 
said phase 1 selection module and said database for selecting a second set of candidate 
finalists, said second set of candidate finalists being a subset of said sub-pool, said first set of 
candidate finalists plus said second set of said candidate finalists equal in number to said 
predetermined total number. 

Claim 8. (Original) A system as in claim 7, said phase 1 selection module comprising: a 
scoring module coupled to said evaluation subsystem and said database for generating said 
total scores; a grouping module coupled to said scoring module and said database for 
grouping said candidate data sets into score tiers responsive to corresponding said total scores, 
each of said score tiers comprising one or more of said candidate data sets having equal said 
total scores; a selection module coupled to said ranking module, said ranking workstation, 
and said database for selecting a first set of candidate finalists responsive to said ranking, said 
first set of candidate finalists equal to or less than said predetermined total number; and an 
output module for outputting said sub-pool, said sub-pool of candidate profiles comprising 
one of said score tiers, said one of said score tiers having the highest total score of those of 
said score tiers containing candidate profiles not selected for said first set of candidate 
profiles. 
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Claim 9. (Original) A system as in claim 7, said phase 2 selection module comprising: a 
receiving submodule coupled to said for receiving said sub-pool of said candidate data sets 
from said phase 1 selection module; an evaluation response selection submodule for 
determining one of said received scores to be weighted and a weight; a weighting submodule 
coupled to said selection module and said evaluation response selection submodule for 
weighting said one of said received scores of said candidate data sets in said sub-pool; a 
scoring submodule coupled to said weighting submodule for generating weighted scores for 
said candidate data sets in said sub-pool; a grouping submodule coupled to said scoring 
submodule for grouping said candidate data sets in said sub-pool into sub-tiers responsive to 
corresponding weighted scores; a selection submodule for selecting additional candidate 
finalists responsive to said weighted evaluation tier ranking; and a stop submodule for 
determining when a correct number of candidate finalists have been selected and outputting 
said additional candidate finalists. 

Claim 10. (Original) A system as in claim 1 wherein said selection system comprises: a 
scoring module for summing evaluation scores for each of said candidate data sets having 
been evaluated; a score grouping module for grouping said candidate data sets having been 
evaluated into score tiers each comprising substantially similar corresponding ones of 
evaluation score sums; a phase 1 selection module for selecting all said candidate data sets of 
the score tiers beginning with the score tier having the highest median evaluation score sum 
and proceeding to score tiers of successively lower median evaluation score sums such that 
the number of candidate documents selected is maximized without exceeding a 
predetermined number, said phase 1 selection module further outputting an active group, said 
active group comprising the candidate data sets of the score tier having the highest median 
evaluation score sum containing no candidates having been selected; and a phase 2 selection 
module for selecting a subset of said active group. 

Claim 1 1 . (Original) A system as in claim 1 0 wherein said phase 2 selection system 
comprises: a weighting module for weighting said evaluation scores for each of said 
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candidate data sets in said active group; a summing module for summing said weighted 
evaluation scores for each of said candidate data sets in said active group; a score grouping 
module for grouping said candidate data sets in said active group into phase 2 score tiers by 
substantially similar corresponding said evaluation score sums; a tie-breaking selection 
module for selecting all candidate documents within the phase 2 score tiers beginning with 
the phase 2 score tier having the highest median evaluation score sum and proceeding to 
phase 2 score tiers of successively lower median evaluation score sums such that the total 
number of selected said candidate data sets is maximized without exceeding said 
predetermined number; and a designating module for designating the candidate data sets of 
the phase 2 score tier having the highest median evaluation score sum containing no 
candidates having been selected as the active group. 

Claim 12. (Original) A system as in claim 1 further comprising: a monitoring subsystem 
coupled to said evaluation subsystem for monitoring evaluations of qualified said candidate 
data sets, said monitoring subsystem allowing interaction with an ongoing evaluation, said 
monitoring subsystem allowing nullification of an evaluation when said evaluation is 
determined to be inconsistent with the requirements of said competition. 

Claim 13. (Original) A system as in claim 1 further comprising: a confirmation subsystem 
coupled to said selection subsystem and said database for determining the confirmation status 
of said one or more competition finalists and determining one or more competition winners 
responsive to said confirmation status determination. 

Claim 14. (Original) A system as in claim 13 wherein said confirmation subsystem 
comprises: a workstation comprising: a display for displaying candidate packages and 
corresponding confirmation documents, and an input device for entering information; a 
confirmation document request module coupled to said database for requesting confirmation 
documents; a confirmation module coupled to said workstation and said database for 
providing said candidate packages and said corresponding confirmation documents to said 
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workstation for display and for receiving confirmation responses; and a conversion module 
coupled to said workstation and said database for marking ones of said candidate data sets as 
non-selected responsive to a corresponding confirmation response indicating nonconfirmation 
and for marking ones of candidate data sets as competition winners responsive to a 
corresponding confirmation response indicating confirmation. 

Claim 15. (Original) A system as in claim 1 further comprising: a competition winner 
tracking subsystem coupled to said database for tracking said competition winners 
comprising: a competition winner tracking module coupled to said database for determining 
when update documents are required for a competition winner, an update document request 
module coupled to said competition-winner module for coordinating update document 
request correspondence, and an update module coupled to said database for updating 
confirmation winner packages with said required update documents. 

Claim 16. (Original) A system as in claim 15, wherein said competition winner tracking 
subsystem further comprises: a workstation comprising: a display for displaying information 
from one of said candidate data sets and corresponding confirmation documents, and an input 
device for entering information; and a reconfirmation module coupled to workstation and said 
database for displaying the candidate data sets of said competition winners with 
corresponding said required documents and receiving reconfirmation responses, the status of 
the candidate data sets of said competition winners being updated in response to said 
reconfirmation responses. 

Claims 17-20 (Cancelled). 

Claim 2 1 . (Currently Amended) A system for evaluating and selecting competition 
winners from a pool of candidates in a competition comprising: 
a database for storing candidate records; 
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a qualification subsystem coupled to said database for determining said candidate 
records 

to be qualified or disqualified; 

a grouping subsystem coupled to said database for grouping said candidate records 
into 

evaluation groups responsive to information contained in said candidate records; 

an evaluation subsystem coupled to said qualification subsystem and said database for 
evaluating and scoring each of said candidate records having been qualified over one or more 
evaluation variables; [[and]] 

a selection subsystem coupled to said evaluation subsystem and database for selecting 
one or more competition finalists for each of said evaluation groups responsive to said 
evaluation and scoring of said candidate records having been qualifie d; and 

a tracking subsystem coupled to said evaluation subsystem for tracking said one or 
more competition finalists and one or more non-selected candidates and determining an 
actual success indicator for each of one or more performance variables . 

Claim 22. (Original) A system as in claim 21, further comprising: an eligibility subsystem 
coupled to said database for determining said competition records to be eligible or ineligible 
responsive to a minimum eligibility standard and information contained in said competition 
records, each of said evaluation groups having a corresponding said minimum eligibility 
standard. 

Claim 23. (Original) A system as in claim 22, wherein said minimum eligibility standard 
varies with the number of candidate profiles. 

Claim 24. (Original) A system for selecting one or more candidates from a plurality of 
candidate profiles comprising: a storage adapted to store a plurality of candidate profiles; an 
eligibility unit coupled to said storage adapted to determine said candidate profiles to be 
eligible or ineligible; an evaluation unit coupled to said storage adapted to facilitate 
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evaluation of said candidate profiles by the scoring of one or more variables by one or more 
evaluators; a first selection unit coupled to said storage adapted to receive said variables 
having been scored, said first selection unit further adapted to select a subset of said 
candidate profiles responsive to said variables having been scored, said subset of said 
candidate profiles not exceeding a predetermined value in number; and a second selection 
unit coupled to said first selection unit and said storage adapted to receive a sub-pool of said 
candidate profiles and adapted to output a subset of said sub-pool of candidate profiles 
responsive to application of predetermined variable weights to said variables having been 
scored, said subset of said candidate profiles added to said subset of said sub-pool of 
candidate profiles equaling said predetermined value in number. 

Claim 25. (Currently Amended) A m e thod system as in claim 24, said first selection unit 
comprising: 

a scoring module coupled to said storage adapted to generate total scores for said 
candidate profiles responsive to said variables; 

a grouping module coupled to said scoring module and said storage for grouping said 
candidate profiles into score tiers responsive to corresponding said total scores, each said 
score tiers comprising one or more said candidate profiles having equal said total scores; 

a selection module coupled to said grouping module and said storage for selecting a 
first set of candidate profiles responsive to said grouping, said first set of candidate profiles 
equal to or less than said predetermined total number, said candidate profiles in said first set 
of candidate profiles having corresponding said total scores higher than said total scores of 
any said candidate profiles not in said first set of candidate profiles; and 

an output module for outputting said sub-pool of candidate profiles, said sub-pool of 
said candidate profiles comprising one of said score tiers, said one of said score tiers having 
the highest total score of those of said score tiers containing candidate profiles not selected 
for said first set of candidate profiles. 
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Claim 26. (Currently Amended) A m e thod system as in claim 24, said second selection 
unit comprising: 

a receiving module coupled to said for receiving said sub-pool of said candidate 
profiles from said phase 1 selection module; 

an evaluation response selection submodule for determining one of said one or more 
evaluation responses to be weighted and a weight, 

a weighting module coupled to said selection module and said evaluation response 
selection submodule for weighting said one of said evaluation responses of candidate data 
sets in said sub-pool, 

a scoring module coupled to said weighting module for generating weighted scores 
for said candidate profiles in said sub-pool; 

a grouping module coupled to said scoring module for grouping said candidate 
profiles in said sub-pool into sub-tiers responsive to corresponding weighted scores, 

a selection module for selecting additional candidate profiles responsive to said 
weighted evaluation tier ranking, and 

a stop module for determining when a correct number of candidate profiles have been 
selected and outputting said additional candidate profiles. 

Claim 27. (Currently Amended) A method for administering a competition system 
comprising: 

receiving one or more candidate packages, each said candidate package containing 
information about a candidate in a competition; 
for each said candidate package: 

determining whether said candidate package meets a minimum application standard, 
selecting said candidate package when said candidate package is determined to meet 
said 

minimum application standard, 
selecting an evaluator, 

enabling said evaluator to evaluate said selected candidate package, [[and]] 
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receiving an evaluation of said selected candidate package from said evaluator; 
ranking all said evaluated candidate packages responsive to corresponding received 
said evaluations; [[and]] 

selecting a predetermined number of competition finalists responsive to said 

calculated 
overall scores!"!",]! ; and 

tracking the performance of said competition finalists and one or more non-selected 
candidates; and determining an actual success indicator for each of said competition finalists 
and said non-selected candidates . 

Claim 28. (Original) The method of claim 27 wherein said step of selecting an evaluator 
comprises: choosing an evaluator; determining whether said evaluator is eligible to evaluate 
said selected candidate package; choosing another evaluator and returning to said step of 
determining whether said evaluator is eligible to evaluate said selected candidate package 
responsive to said determination of whether said evaluator is eligible to evaluate said selected 
candidate package indicating said evaluator is not eligible to evaluate said selected candidate 
package; and selecting said evaluator responsive to said determination indicating said 
evaluator is eligible to evaluate said selected candidate package. 

Claim 29. (Original) The method as in claim 28 wherein the step of determining whether 
said evaluator is eligible to evaluate said candidate package comprises: sending an evaluator 
eligibility question to said evaluator; receiving a response to said evaluator eligibility 
question from said evaluator; and determining whether said evaluator is eligible to evaluate 
said queued candidate package responsive to said received response regarding said evaluator 
eligibility question. 

Claim 30. (Original) The method as in claim 27, wherein said step of receiving a plurality of 
candidate packages, each said candidate package containing information about a candidate in 
a competition comprises: receiving a plurality of application documents each relating to a 

1 1 Serial No. 09/996,907 

Attorney Docket No. Docket No.: 570862000100 



candidate; and grouping said application documents such that all said application documents 
relating to the same candidate are grouped into a candidate package. 

Claim 3 1 . (Original) The method of claim 27, said method further comprising: determining 
whether said selected candidate package is eligible to be evaluated; and denying said 
evaluator from evaluating said selected candidate package responsive to said determination 
indicating said selected candidate package is not eligible to be evaluated. 

Claim 32. (Original) The method as in claim 27 wherein the step of determining whether 
said candidate package is eligible to be evaluated comprises: sending a candidate package 
eligibility question to said evaluator; receiving a response from said evaluator regarding said 
candidate package eligibility question; and determining whether said candidate package is 
eligible for evaluation responsive to said received response regarding said candidate package 
eligibility question. 

Claim 33. (Original) A method as in claim 27 wherein said one or more minimum 
application standards comprises: one or more minimum response standards; one or more 
eligibility requirements; and one or more qualification requirements. 

Claim 34. (Original) A method as in claim 27 further comprising: determining whether any 
of said competition finalists are confirmed; and selecting a new competition finalist for each 
competition finalist determined not to be confirmed and returning to said step of determining 
whether any of said competition finalists are confirmed. 

Claim 35. (Original) A method for selecting one or more candidates from a plurality of 
candidate applications in a competition system, comprising: determining for each of said 
candidate applications whether said candidate application meets minimum qualification 
standards; determining the number of said candidate applications meeting said minimum 
qualification standards; determining for each said candidate application meeting said 
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minimum qualification standards whether said candidate application meets a discretionary 
qualification standard, said discretionary qualification standard being a function of said 
determined number of said candidate applications meeting said minimum qualification 
standards; evaluating each said candidate application determined to meet said minimum 
qualification standards; scoring each of said candidate applications that has been evaluated 
over a set of evaluation variables; and selecting one or more of said candidates as competition 
finalists responsive to said scoring. 

Claim 36. (Original) A method as in claim 35 wherein said step of determining for each said 
candidate application whether said candidate application meets minimum qualification 
standards comprises: i) filtering each candidate's candidate information to determine whether 
said candidate's candidate information meets at least one of an information completeness 
standard and a document completeness standard; ii) filtering said candidate information to 
ensure said candidate information indicates said candidate meets one or more minimum 
qualification requirements; iii) grouping said candidate information into two or more groups 
responsive to one or more predetermined pieces of information contained in said candidate 
information; and iv) filtering said candidate information to ensure said candidate information 
indicates said candidate meets one or more group level standards. 

Claim 37. (Original) A method as in claim 35 wherein said step of filtering said candidate 
information to ensure said candidate information indicates said candidate meets one or more 
minimum qualification requirements comprises requiring each said candidate's candidate 
information includes at least a candidate application, a nomination form, and a 
recommendation form. 

Claim 38. (Original) A method as in claim 35 wherein said step of filtering said candidate 
information to ensure said candidate information indicates said candidate meets one or more 
group level standards comprises passing said candidate when said candidate information 
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meets or exceeds at least one of a minimum GPA score, a family income requirement, and 
completion of a GED. 

Claim 39. (Original) A method as in claim 35 wherein said step of grouping comprises 
grouping said candidates by one or more of the following candidate information: 
geographical residence, urban/rural upbringing, primary ethnicity, and cultural background. 

Claim 40. (Original) A method as in claim 35 wherein said step of filtering said candidate 
information to ensure said candidate information indicates said candidate meets one or more 
group level standards comprises passing said candidate when said candidate information 
indicates said candidate meets or exceeds a third filter set, said third filter set comprising at 
least one of a minimum number of academic awards, a minimum number of public awards, a 
minimum number of honors, a minimum number of leadership roles, a minimum number of 
student excel bubbles completed, a minimum class rigor average, a minimum amount of 
community service, a maximum amount of personal circumstance exceptions, and a 
minimum number of paid hours of employment. 

Claim 41 . (Original) A method as in claim 35 wherein said step of evaluating comprises the 
steps of: matching each said candidate information with a reader qualified to read said 
candidate information, a reader being disqualified from reading a particular candidate's 
candidate information when any conflicts exist; and reading each said candidate information 
by said reader, said step of reading comprising scoring said candidate information on a set of 
reading variables. 

Claim 42. (Currently Amended) A method as in claim 41 wherein said set of reading 
variables includes th e following non-cognitive variables , said non-cognitive variables 
comprising rnil apositive self-concept variable , a realistic self-appraisal variable , an 
understanding/navigation of a social system variable , a preference of long-term goals over 
short-term goals variable , an availability of a strong support person variable , a leadership 
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experience variable , a community service variable , and an interest/knowledge in a non-school 
field variable . 

Claim 43. (Original) A method as in claim 35 wherein said step of selecting comprises the 
steps of: summing said evaluation variable scores for each of said candidate applications to 
produce a total reading variable score; ranking said candidate applications by their respective 
total reading variable score; determining the highest score tier having ones of said candidate 
applications not already selected; determining whether the sum of all said candidate 
applications already selected plus the number of said candidate applications at the current 
score tier exceeds the total number of said candidate applications to be selected; when the 
sum of all said candidate applications already selected plus the number of said candidate 
applications at the current score tier is less than the total number of said candidate 
applications to be selected, selecting the candidate applications in the current score tier and 
continuing back to said step of determining the highest score tier; when the sum of all said 
candidate applications already selected plus the number of said candidate applications at the 
current score tier equals the total number of said candidate applications to be selected, 
selecting the candidate applications at the current score tier and aborting any further 
selection; when the sum of all said candidate applications already selected plus the number of 
said candidate applications at the current score tier exceeds the total number of said candidate 
applications to be selected, carrying out the steps of: determining the next highest score tier 
having one or more said candidate applications not already selected, selecting one of said set 
of evaluation variables; altering the weight of said selected evaluation variable over all 
unselected said candidate applications in the score tier determined to be the next highest to 
produce weighted sets of reading scores, summing each said candidate applications weighted 
evaluation variable scores to produce a total evaluation variable score; grouping said 
candidate applications by their respective total weighted reading variable scores; beginning 
with the highest ranking total weighted reading variable score group and continuing 
successively to the next lower total weighted reading variable score group, selecting the 
candidate applications of as many total weighted evaluation variable score groups as possible 
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without exceeding the total number of said candidate applications to be selected, and 
selecting a previously-unselected evaluation variable from said set of evaluation variables 
and returning to said step of altering the weight of said selected evaluation variable and 
continuing. 

Claim 44. (Currently Amended) A method as in claim 35 whoroin said st e p of further 
comprising requesting confirmation documents from one or more of said competition finalists. 
confirming compris e s r e quiring e ach said candidat e 's candidate information to include a 
sp e cification of citizenship or primary residency, a tribal docum e nt when said candidat e is 
American Indian, information r e garding community servic e , information r e garding 
e mploym e nt, information r e garding honors and awards, and information r e garding l e ad e rship 
e xp e ri e nc e . 

Claims 45-53 (Cancelled). 

Claim 54. (Original) A method of reducing the number of candidates in a candidate pool of 

a competition system, each candidate being represented in said competition system by a 

corresponding candidate information package, said method comprising: determining whether 

each candidate information package in said candidate pool meets an information 

completeness standard and a document completeness standard; removing all candidate 

information packages from said candidate pool which do not meet said information 

completeness standard and said document completeness standard; determining whether each 

said candidate information package in said candidate pool indicates the corresponding 

candidate meets one or more minimum qualification standards; removing all candidate 

information packages from said candidate pool which do not indicate the corresponding 

candidate meets said minimum qualification standards; grouping said candidate information 

packages in said candidate pool into one or more groups based on information contained in 

said candidate information packages; determining whether said candidate information 

packages in said candidate pool indicate the corresponding candidate meets one or more 
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discretionary standards; and removing all candidate information packages from said 
candidate pool which do not indicate the corresponding candidate meets said one or more 
discretionary standards. 

Claim 55. (Original) A method as in claim 54 wherein said step of determining whether 
each candidate information package in said candidate pool meets an information 
completeness standard and a document completeness standard comprises determining 
whether each said candidate information package includes at least a candidate application, a 
nomination form, and a recommendation form. 

Claim 56. (Original) A method as in claim 54 wherein said step of determining whether 
each said candidate information package in said candidate pool indicates the corresponding 
candidate meets one or more minimum qualification standards comprises passing said 
candidate when said candidate information meets or exceeds at least one of a minimum GPA 
score, a family income requirement, and completion of a GED. 

Claim 57. (Original) A method as in claim 54 wherein said step of grouping comprises 
grouping said candidates by one or more of geographical residence, urban/rural upbringing, 
primary ethnicity, and cultural background. 

Claim 58. (Original) A method as in claim 54 wherein said step of determining whether said 
candidate information packages in said candidate pool indicate the corresponding candidate 
meets one or more discretionary standards comprises determining whether said candidate 
information indicates said candidate meets or exceeds a third filter set, said third filter set 
comprising at least one of a minimum number of academic awards, a minimum number of 
public awards, a minimum number of honors, a minimum number of leadership roles, a 
minimum number of student excel bubbles completed, a minimum class rigor average, a 
minimum amount of community service, a maximum amount of personal circumstance 
exceptions, and a minimum number of paid hours of employment. 
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Claim 59. (Original) A method as in claim 54 wherein said step of determining whether said 
candidate information packages in said candidate pool indicate the corresponding candidate 
meets one or more discretionary standards comprises determining whether said candidate 
information indicates said candidate meets or exceeds a minimum number of academic 
awards, a minimum number of public awards, a minimum number of honors, a minimum 
number of leadership roles, a minimum number of student excel bubbles completed, a 
minimum class rigor average, a minimum amount of community service, a maximum amount 
of personal circumstance exceptions, and a minimum number of paid hours of employment. 

Claims 60-66 (Cancelled). 

Claim 67. (Currently Amended) A method of monitoring the evaluation of individuals in 
a pool of candidates in a competition system comprising: 

receiving one or more evaluator responses for an evaluation from a first evaluator; 

analyzing said evaluation responses; and 

determining whether said evaluation should be nullified based on said analysis . 
receiving data from an e valuation; d e t e rmining from said data an e valuation ind e x; 
comparing said e valuation ind e x to corresponding r e ference indioioa and determining wh e ther 
said e valuation is ab e rrant r e sponsiv e to said comparison . 

Claim 68. (Currently Amended) A method as in claim [[65]] 67, wherein said analysis 
comprises comparing at least one of said evaluator responses to one or more evaluator 
responses given by said evaluator in at least one previously completed evaluation, evaluation 
ind e x compris e s on e or more evaluation r e spons e s and said corr e sponding r e f e r e nc e indioi e s 
comprise one or more of: the average corresponding e valuation r e spons e s of said e valuator 
for th e corresponding e valuation r e sponses ov e r two or mor e prior compl e t e d e valuations and 
th e av e rag e corr e sponding e valuation r e spons e s for on e or more other ovaluators in said 
comp e tition syst e m ov e r two or mor e prior compl e t e d e valuations. 
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Claim 69. (Currently Amended) A method as in claim 67, wherein said analysis 
comprises comparing a ovaluation ind e x comprises tho number of reference sources checked 
per evaluator e valuation response to and said corr e sponding r e f e r e nc e indici e s compris e a 
minimum number of sources to be checked per evaluator e valuation response. 

Claim 70. (Currently Amended) A method as in claim 67, wherein said analysis 
comprises comparing an e valuation ind e x compris e s th e average time between said evaluator 
responses given in said evaluatio n and said corr e sponding reference indicies compris e on e or 
mor e of: th e to an av e rag e corr e sponding average time between prior evaluator responses of 
said evaluator ever given in at least one two or mor e prior completed evaluation, e valuations 
and th e av e rag e corresponding av e rag e tim e b e tw ee n responses for one or mor e oth e r 
e valuators in said comp e tition syst e m ov e r two or mor e prior compl e t e d e valuations. 

Claim 71 (Cancelled). 

Claim 72. (Original) A method of ensuring fair and consistent evaluation comprising: 
providing an evaluation workstation for use by an evaluator; making available 
candidate 

applications from one or more candidates needing evaluation; 

matching said evaluator to one or more of said candidate applications; 

receiving a request from said evaluator to evaluate one of said one or more candidate 
applications; 

providing one or more eligibility questions to said evaluator; 
receiving responses to said eligibility questions; 

determining from said responses to said eligibility questions whether said evaluator is 
eligible to evaluate said one candidate application; 

preventing said evaluator from evaluating said candidate application responsive to 
said 
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first determination indicating said evaluator is not eligible to evaluate said candidate 
application; 

providing one or more candidate eligibility questions to said evaluator; 
receiving responses to said candidate eligibility questions; 

determining from said responses to said candidate eligibility questions whether said 
candidate is eligible to be evaluated; 

preventing said evaluator from evaluating said candidate application responsive to 
said second determination indicating said candidate is not eligible to be evaluated; 

presenting in a substantially simultaneous manner, said candidate application, an 
evaluation rubric, and a scoring display to said evaluator; 

monitoring the evaluation of said candidate package by said evaluator; and 

interacting with said evaluation responsive to the determination that said evaluation is 
unfair or inconsistent. 

Claim 73. (Currently Amended) A method as in claim 72 wherein said step of 
monitoring includes at least one of detecting whether the average time between question 
responses is less than a minimum time period, d e t e rmination determining whether the 
average time between question responses is less than the average time period between 
question responses for said evaluator during past evaluations, determining whether the 
average evaluation score is above a maximum score, determining whether the average 
evaluation score is below a minimum score, and determination whether any evaluation score 
is inconsistent with information in said candidate application. 

Claim 74. (Original) A method as in claim 72 wherein said step of interacting comprises at 
least one of terminating said evaluation, initiating a conversation with said evaluator, and 
providing an automated message to said evaluator. 

Claims 75- 90 (Cancelled). 
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Claim 91 . (New) A method as in claim 67 wherein said analysis comprises comparing at 
least one of said evaluator responses given in said evaluation to one or more evaluator 
responses given by a second evaluator. 

Claim 92. (New) A method as in claim 67, wherein said analysis comprises comparing 
an average time between said evaluator responses given in said evaluation to an average time 
between evaluator responses for one or more evaluator responses given by a second evaluator. 

Claim 93 (New) A system as in claim 1, further comprising a tracking subsystem 
coupled to said evaluation subsystem for tracking said one or more competition finalists and 
one or more non-selected candidates and determining an actual success indicator for each of 
one or more performance variables. 
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